Introduction to CNNs and LSTMs for NLP

I put together these notes as part of my TA work for the Graph and Text Mining grad course of Prof. Michalis Vazirgiannis in the Spring of 2017. They accompanied a programming lab session about Convolutional Neural Networks (CNNs) and Long Short Term Memory networks (LSTMs) for document classification, using Python and Keras1. Keras is a very popular Python library for deep learning. It is a wrapper for TensorFlow, Theano, and CNTK. To write this handout, I curated information mainly from: the original 2D CNN paper [11] and Stanford’s CS231n CNN course notes, Zhang and Wallace practitioners’ guide to CNNs in NLP [18], the seminal papers on CNN for text classification [8, 9]; Denny Britz’ tutorial on RNNs, and Chris Colah’s post on understanding LSTMs. Last but not least, Yoav Golderg’s primer on neural networks for NLP [5] proved very useful in understanding both CNNs and RNNs. The CNN part of the code can be found on my GitHub here.